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(54) Multimedia description scheme having weight information and method for displaying 
multimedia 



(57) The present invention relates to a multimedia 
Indexing using a weight description scheme, and more 
parttcularty, to a weight description scheme and a mul- 
timedia indexing method using the same which is capa- 
ble of increasing the perfomiance of multimedia index- 
ing and browsing by effectively describing weight infor- 
mation on multimedia items. The present invention re- 



fers to a weight description scheme and multimedia in- 
dexing method using the same, and provides a multime- 
dia description scheme for enabling effective multime- 
dia indexing and browsing by having a weight descrip- 
tion scheme consisting of a weight information type, 
weight information, and its reliability so that each de- 
scription scheme representing content information of 
multimedia describes effective weight information. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0001] The present invention relates to a multimedia 
indexing using a weight description scheme, and more 
particularly, to a weight description scheme(hereinafter, 
"weight DS") and a multimedia indexing method using 
the same which is capable of increasing the perform- 
ance of multimedia indexing and browsing by effectively 
describing weight information on multimedia items. 

2. Description of the Background Art 

[0002] For content-based image indexing, various 
feature descriptors such as a color histogram, partial 
section colors etc. are used. However, since a main fea- 
ture descriptor for identifying an image varies according 
to the image, many techniques for adapting weight In- 
formation on the feature descriptor corresponding to the 
feature of the Image have been provided. 
[0003] As a technique for adapting weight information 
corresponding to the feature of an image In a system for 
content-based image indexing according to the conven- 
tional art, there is a technique for preparing a window 
for inputting weight information at an user interface so 
that an user sets weight information on a feature de- 
scriptor during indexing, thus making the user input 
weight Information on the feature descriptor using the 
interface during the Indexing. 
[0004] However, such a method has a problem that it 
is difficult for a general user to set a weight information 
on a feature descriptor corresponding to the image. 
[0005] As another technique for describing weight in* 
formation for content-based image indexing according 
to the conventional art. a method for setting weight in- 
formation on a feature descriptor using user relevance 
feedback is employed. 

[0006] In this method, the user firstly perfomns index- 
ing once in the present state, then marks whether or not 
an Image is similar to which the user searches for in the 
list according to the result of the indexing, and then pro- 
vide feedback to a indexing system, thus making the 
system automatically set or update weight information 
using the feedback. 

[0007] This method is advantageous in that the sys- 
tem itself automatically sets weight information on a fea- 
ture descriptor by using the user's feedback. 
[0008] The above-described techniques according to 
the conventional art shows examples of an weighting 
scheme adapted to a technique for a certain type of mul- 
timedia, such as an image. 

[0009] However, since demands for content-based in- 
dexing and browsing of more complicated multimedia 
are increased recently, there arise a need for a weight- 
ing scheme for integrated multimedia indexing and 



browsing in order to meet these demands. 

SUMMARY OF THE INVENTION 

5 [0010] Accordingly, it is an object of the present inven- 
tion to enable effective multimedia indexing and brows- 
ing by providing an Integrated multimedia structure and 
an efficient weight description scheme according to the 
structure. 

10 [0011] The present invention is a weight description 
scheme and multimedia indexing method using the 
same, and provides a multimedia description scheme 
for enabling effective multimedia indexing and browsing 
by having a weight DS consisting of a weight informatiorr 

IS type, weight information, and its reliability so that each 
description scheme representing content information of 
multimedia describes effective weight information. 
[0012] In addition, the present invention is a weight 
description scheme and multimedia Indexing method 

20 using the same, and provides a multimedia DS for ena- 
bling effective Indexing and browsing by describing 
weight information on segments or weight information 
on segment relations in such a manner that a multimedia 
content DS Includes a segment DS and a segment re- 

25 lational graph DS, said multimedia content OS further 
including more than one weight DS. 
[0013] In addition, the present invention is a weight 
description scheme and multimedia indexing method 
using the same, and provides a multimedia DS for ena- 

30 bling effective Indexing and browsing by describing 
weight infonnatton on segment relations or weight Infor- 
mation on segment nodes in such a manner that a mul- 
timedia content DS has a segment relational graph DS 
consisting of more than one segment relation DS and 

35 more than one segment node DS. said one segment re- 
lation DS including more than one segment relation DS 
or more than one segment node, said one segment 
node DS including more than one segment or segment 
node DS. or more than one segment relation DS, said 

^ segment relation DS further Including more than one 
weight DS, and by describing weight Information on seg- 
ments In such a manner that the segment node DS fur- 
ther includes more than one weight DS. 
[0014] In addition, the present invention provides a 

^5 multimedia DS for enabling effective multimedia index- 
ing and browsing by describing weight information on 
objects, weight Information on events, or weight infor- 
mation on event/object relational graphs in such a man- 
ner that a multimedia content DS includes a content DS 

50 consisting of a plurality of event information, a plurality 
of object information, and a plurality of event/object re- 
lational graph DSs, said content DS further including 
more than one weight DS. 

[0015] In addition, the present invention provides a 
55 multimedia DS for enabling effective multimedia index- 
ing and browsing by describing weight information on 
relations between events and objects or weight informa- 
tion on event or object nodes in such a manner that a 
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multimedia event/object relational graph DS further in- 
cludes more than one weight DS, and by describing 
weight information on events or objects in such a man- 
ner that an event or object node DS further includes 
more than one weight DS. 

[0016] in addition, the present invention provides a 
method of displaying multimedia which makes it possi- 
ble to perform a multimedia summary display using a 
multimedia DS. weight DS. and segment relational 
graph DS. 

[0017] In addition, the present invention provides a 
method of displaying multimedia which makes it possi- 
ble to display moving images by indexing the same by 
character and relation using a multimedia DS and con- 
tent information, weight DS. and event/object relational 
graph DS. 

[0018] Additional advantages, objects and features of 
the invention wilt become more apparent from the de- 
scription which follows. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0019] The present invention will become better un- 
derstood with reference to the accompanying drawings 
which are given only by way of illustration and thus are 
not limitative of the present invention, wherein: 

Figures 1 A and 1 B are views illustrating a common- 
ly used weight description scheme(DS). wherein 
Figure 1A illustrates the weight DS, and Figure IB 
describes the same in a XML format; 
Figure 2 is a view illustrating a segment DS; 
Figures 3A and 3B are views illustrating examples 
of a description of weight information on segments 
in the segment DS of Rgure 2; 
Figures 4A and 4B are views illustrating a multime- 
dia syntactic DS with a weight DS, wherein Figure 
4A illustrates the syntactic DS. and Figure 4B de- 
scribes the same in the XML format: 
Figures 5A and 5B are views illustrating a segment 
relational graph DS with a weight DS. wherein Fig- 
ure 5A illustrates the segment relational graph DS, 
and Figure 5B describes the same in the XML for- 
mat; 

Figures 6A and 6B are views illustrating a multime- 
dia semantic DS with a weight DS, wherein Figure 
6A illustrates the multimedia semantic DS, and Fig- 
ure 68 describes the same in the XML format; 
Figures 7A and 7B are views illustrating an event/ 
object relational graph DS with a weight DS. where- 
in Figure 7A illustrates the event/object relational 
graph DS, and Figure 7B describes the same in the 
XML format; 

Figures 8A and 8B are views illustrating a browsing 
in a key-frame transition graph format using the 
segment DS and the segment relational graph DS 
within the multimedia syntactic DS; 
Figure 9 is a view illustrating an example of a char- 



acter/relation-based moving image browser. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

5 

[0020] Figures 1 A and 1 B illustrate a weight DS com- 
monly used in the present invention, wherein Figure 1 A 
illustrates the weight DS, and Figure 16 describes the 
same in a XML format. 

10 [0021] The multimedia weight DS of the present in- 
vention in Figure 1 includes a weight type, weight value 
DS, and reliability so that each DS representing the in- 
formation of multimedia content can effectively describe 
weight information. 

15 [0022] As seen from Figure 1A. a weight DS 101 in- 
cludes a weight type 102, weight value DS 103, and re- 
liability 104. 

[0023] The weight type 1 02 means the criteria for de- 
fining a weight value. The same object can have a plu- 

20 rality of weight inform according to its criteria. 

[0024] For example, as weight information on feature 
descriptors used in image indexing are described, two 
kinds of weight information can be simultaneously de- 
scribed with the respect of the same object, e.g.. one is 

25 a weight information generated by leaming through the 
user's feedback, and the other is a weight infonmation 
directly designated by an image generator. 
[0025] At this time, these two kinds of weight informa- 
tion can be distinguished from each other according to 

30 their generation using the weight type 102. 

[0026] For example, the weight type of the former one 
(weight information generated by leaming through the 
user's feedback) is "relevance feedback", and the latter 
one(weight information directly designated by the image 

S5 generator) is "manuar. 

(0027] In addition, the reliability 104 shows what the 
weight information assigned to an object according to 
the weight type 103 is a reliable value. In other words, 
it shows if the cunrent weight infonmation is reliable. 

40 [0028] For example, if the weight type is "relevance 
feedback", the reliability can be assigned according to 
how much feedback is provided, or if the weight type is 
"manual", the reliability of a k>w value or high value can 
be assigned respectively to whether the person assign- 

45 ing weight information is a naive user or expert with re- 
spect to the conresponding object. 
[0029] In Figures 1A and IB. the weight DS 103 in- 
cludes a plurality of weight value DSs each consisting 
of a reference 105 and a weight value 106. 

50 [0030] For example. Figure 2 is a view illustrating a 
segment DS. and Figures 3A and 3B are views illustrat- 
ing examples of a description of weight infonmation on 
segments in the segment DS of Figure 2. As shown in 
Figures 3 A and 3B. the weight information on segments 

55 in the segment DS are described as a series of weight 
value DS references and weight values respectively re- 
ferring to the segments. 

[0031 ] tn other words, with respect to each of the seg- 
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ment DSs 201 through 201 . weight information is de- 
scribed as a series of weight value DS references and 
weight values referring to each segment DS. Figure 3A 
describes weight information of all the segments includ- 
ed in a hierarchical structure, and Figure 3B describes 5 
the weight infonmation thereof included in the hierarchi- 
cal structure in unit of segment group by each level. 
[0032] As an example of the description of the weight 
information on the segments as a series of weight ob- 
jects and weight values referring to each segment as 
shown in Figure 3 in the segment DSs of Figure 2, seg- 
ment O(segO) 201 is described as a weight value DS ref- 
erence(=*segO*) and a weight value(=0.6), and other 
segments are also described as a weight value DS ref- 
erence and a weight value. 

[0033] However, in the case that the order of a plural- 
ity of weight value DS references are previously defined, 
weight information can be described in a n-dimensional 
vector consisting of only weight values, but no informa- 
tion on weight value DS references. 
[0034] At this time, since each weight value can be 
mapped to the corresponding weight value DS refer- 
ence according to the previously defined order, the 
weight value DS references can be omitted. 
[0035] In addition, in the case that the order of the 
weight value DS references are not previously defined, 
weight information can be described as a set of pairs of 
weight value DS references and their weight values. 
[0036] Figures 4A and 4B are views illustrating a mul- 
timedia syntactic DS with a weight DS. wherein Figure 
4A illustrates the syntactic DS. and Figure 4B descrit>es 
the same in the XML format. 

[0037] The multimedia weight DS of the present in- 
vention as in Figures 4A and 48 includes a syntactic DS. 
which is a multimedia content DS associated with a mul- 
timedia DS. for multimedia content-based indexing and 
browsing. When the syntactic DS includes a segment 
DS representing description schemes of segments and 
a segment-relation graph DS defining the relation be- 
tween the segments, the syntactic DS describes weight 
information on the segments or weight infonmation on 
the relation between the segments by further including 
more than one weight DS. 

[0038] The syntactic DS 401 includes a weight DS 
402. a plurality of segment DSs 403. and a segment- 
relation graph DS 404. 

[0039] The weight DS 402 describes weight informa- 
tion on the segments or the segment-relation graph. 
[0040] The segment DS 403 can be constructed hier- 
archically as shown in Figure 2. At this time, the weight 
DS can be described as weight information of all seg- 
ments included in the hierarchical structure as shown in 
Figure 3A, or can be described as weight information 
thereof included in the hierarchical structure in unit of 
segment group by each level as shown in Figure 38, 
[0041] In other words, as the segment DS describes 
a hierarchical segment description scheme by having 
another child segment DS. one weight DS included in 
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the syntactic DS can describe weight information of all 
segments described in the hierarchical segment DS, or 
each of a plurality- of weight DSs can describe weight 
information of all segments in unit of child segment of 
one segment included in the hierarchical segment DS. 
[0042] In this example, because there exist three seg- 
ment groups in the hierarchical structure, the weight in- 
formation of all segments are described in three weight 
DSs. 

[0043] In this way, various forms of weight information 
can be described using a reference for the target of 
weighting. 

[0044] Figures 5A and 5B are views illustrating a seg- 
ment relational graph DS with a weight DS. wherein Fig- 
ure 5A illustrates the segment relational graph DS. and 
Figure 58 describes the same in the XML format. 
[0045] The multimedia DS of the present Invention in 
Figures 5 A and 5B describes the relation between seg- 
ments by describing weight information on segment re- 
lations or weight information on segment nodes in such 
a manner that a multimedia content DS has a segment 
relational graph DS consisting of more than one seg- 
ment relation DS and more than one segment node DS, 
said one segment relation DS including more than one 
segment relation DS or more than one segment node, 
said one segment node DS including more than one 
segment or segment node DS, or more than one seg- 
ment relation DS, said segment relation DS further in- 
cluding more than one weight DS, and by describing 
weight information on segments in such a manner that 
the segment node DS further includes more than one 
weight DS. 

[0046] The segment relational graph DS 501 includes 
more than one segment relation DS 502 and more than 
one segment node DS 504, said segment relation DS 
502 including more than one segment relation DS or 
segment node, and said segment node DS 504 includ- 
ing more than one segment, segment node DS, or seg- 
ment relation DS and having a reference to region 505, 
conresponding regions 506, segment 507. and refer- 
ence to segment 508. 

[0047] Using this DS, a hierarchical segment relation- 
al graph can be generated. 

[0046] Here, the weight DS 503 included in the seg- 
ment relational graph DS describes weight information 
on the relations between segments, or weight informa- 
tion on segment nodes, and the weight DS 509 included 
in the segment node DS 504 describes weight informa- 
tion on segments included in the relational graph. 
[0049] Figures 6A and 68 are views illustrating a mul- 
timedia semantic DS with a weight DS. wherein Figure 
6A illustrates the multimedia semantic DS. and Figure 
6B describes the same in the XML format. 
[0050] The multimedia DS of the present invention in 
Figures 6 A and 6B describes weight information on ob- 
jects, weight information on events, or weight informa- 
tion on event/object relational graphs for enabling effec- 
tive multimedia indexing and browsing in such a manner 
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that a multimedia content DS includes a semantic DS 
related with multimedia contents, said semantic DS con- 
sisting of a plurality of event information, a plurality of 
object information, and a plurality of event/object rela- 
tional graph DSs, said semantic DS further including 
more than one weight DS. 

[0051] The semantic DS 601 includes a weight DS 6. 
a plurality of object DSs 603. event DS 604. and event/ 
object relational graph DS 605. 
[0052] The weight DS 602 describes weight informa- 
tion on objects, weight information on events, or weight 
Information on event/object relational graphs. 
[0053] The object DS 603. event DS 604. and event/ 
object relational graph 605 can be described hierarchi- 
cally in the same way as in the segment DS and seg- 
ment relational graph DS. At this time. also, weight in- 
formation can be described in unit of parts of each hier- 
archicai stiucture or in the entire hierarchical structure 
using a reference for the target of weighting. 
[0054] In other words, in the multimedia DS of Figures 
6A and 6B, as the object(or event) DS describes a hier- 
archical object(or event) DS by including another child 
object(or event) DS, one weight DS included in the se- 
mantic DS can describe weight information of ail objects 
(or events) described in the hierarchical object(or) DS, 
or each of a plurality of weight DSs can describe weight 
Information thereof in unit of child object(or event) of one 
ob]ect(or event) Included in the hierarchical object(or 
event) structure. 

[0055] Figures 7A and 7B are views illustrating an 
event/object relational graph DS with a weight DS, 
wherein Figure 7A illustrates the event/object relational 
graph DS. and Figure 7B describes the same in the XML 
format. 

[0056] The multimedia DS of the present invention in 
Figures 7A and 8 describes weight information on 
event/object relations or event/object nodes In such a 
manner that and event/object relational graph DS fur- 
ther includes more than one weight DS, and describes 
weight information on events or objects in such a man- 
ner that an event/object node DS further includes more 
than one weight DS. in order to describe the relation be- 
tween the objects and the events, when an event/object 
relation graph DS includes more than one event/object 
relation DS and more than one event/object node DS. 
said event/object relation DS including more than one 
event/relation graph DS or event/object node, and said 
event/object node DS including more than one object, 
event, event/object node DS, or event/object relation 
DS. 

[0057] The basic DS in Figures 7A and 78 is similar 
to the segment relational graph DS described in Figures 
5A and 58. 

[0058] The event/object relational graph DS 701 In- 
cludes more than one event/object relation DS 702. 
weight DS 703. event/object node DS 704. said event/ 
object relation DS 702 including more than one event/ 
object relation DS or event/object node, and said event/ 



object node DS 704 including more than one object, 
event, event/object node DS. or event/object relation 
DS. The event/object node DS 704 consists of a refer- 
ence to object DS 705. object DS 706. event 707. refer- 

5 ence to event 708, and weight DS 709. 

[0059] Here, the weight DS 703 included in the event/ 
object relational graph DS 701 describes weight infor- 
mation on event/object relations or weight information 
on event/object nodes, and the weight DS 709 included 

10 in the event/object node DS 704 describes weight infor- 
mation on events or objects. 

[0060] Meanwhile, as described above, when the 
weight information included in the multimedia content 
DS having a weight DS is a weight information described- 

IS with respect to a specific part of the hierarchical struc- 
ture, or a weight information described with respect to 
one of many items, such as object, event, etc., it is iden- 
tified by means of a weight value DS reference included 
in the weight DS whether it Is a weight information on a 

20 certain part, or a weight infonmation on a certain item. 
[0061] As the above-described multimedia content 
DS. a method for browsing and indexing multimedia 
having weight information, and method for displaying 
multimedia will now be described. 

25 [0062] First, in case of using the multimedia DS and 
segment relational graph DS including a weight DS, a 
key-frame transition graph type browsing as an itser-de- 
pendent key frame-based summarization application 
will be described. 

30 [0063] In displaying a key frame transition graph using 
the above-described weight DS. multimedia content DS. 
and segment relational graph DS. the method for dis- 
playing a multimedia summary including the multimedia 
DS of the present invention includes the steps of: ex- 

35 tracting key frames as much as an user wants in the 
order of weight information on segments described in 
the weight DS In the multimedia content DS; and ex- 
tracting only relations Including the key frames extracted 
In the segment relational graph DS. and generating a 

40 key-frame transition graph using the same. 

[0064] In addition . in displaying a key-frame transition 
graph using the above-described weight DS, multimedia 
content DS. and segment relational graph DS, the meth- 
od for displaying a multimedia summary including the 
multimedia DS of the present invention includes the 
steps of: extracting segment relations In the order of the 
level of weight information on segment relations de- 
scribed in the weight DS in the segment relational graph 
DS until the number of key frames that the user wants 

50 is reached; and generating a key-frame transition graph 
using the extracted segment relational graphs. 
[0065] Hereinafter, the method for displaying a multi- 
media summary will be described with reference to Fig- 
ures 8A and 88. 

55 [0066] The segment means a predetermined portion 
of a moving image on the basis of time axis. The moving 
image consists of continuous frames. Assuming that the 
moving image consists of 30 frames for one second, a 
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segment having a size of two seconds will consists of 
continuous 60 franies. Thus, if the time interval of the 
segment Is 0. it means one frame. Therefore, the seg- 
ment DS may describe one frame. 
[0067] By this characteristic, a hierarchical key-frame 
structure can be described using a segment OS in the 
multimedia content DS, and a key-frame relational 
graph can be described using a segment relational 
graph structure. In particular, a key-frame relation 
means the relation of temporal order between key 
frames, a temporal transition graph can be described. 
[0068] In other words, using the segment DS and seg- 
ment relational graph DS in the multimedia content DS. 
a key-frame transition graph type browsing as shown in 
Figures 8A and SB are made possible. 
[0069] At this time, the user may want to adjust the 
number of nodes of the key-frame transition graph, be- 
cause the complicity of the key-frame transition graph 
disturbs in understanding contents. 
[0070] That is, by means of the same DS, a transition 
graph using five key frames(frame 0 — frame 4) as in 
Figure 8A, or a transition graph using 1 0 f rames(frame 
0. frame 0.1. frame 1. frame i.i, frame 2. frame 2.1. 
frame 3, frarne 3.1 , frame 3.2, frame 4) as in Figure 8B 
can be requested. 

[0071] When such a request Is occurred, a key-frame 
transition graph can be generated by means of key 
frames as much as the user wants In the order of the 
level of weight information, by using the weight DS in- 
cluded In the DS. 

[0072] In the same way. in the case that the user des- 
ignates the approximate number of key frames, a key 
frame transition graph can be generated by extracting 
relations as much as the number of key frames that the 
user wants using the weight DS included In the segment 
relational graph DS in the order of the level of weight 
information, and using the same. 
[0073] At this time, since the extraction is done in the 
order of the level of not weight information on key 
frames, but weight information on key-frame transition, 
it is advantageous in that a key frame transition graph 
more effectively reflecting the overall flow of contents is 
provided. However. It is disadvantageous In that the 
number of frames that the user wants cannot be accu- 
rately reflected. 

[0074] Next, a character/relaiion-based moving im- 
age browser using the multimedia semantic DS and 
event/object relational graph DS including a weight DS 
will be described. 

[0075] In Indexing moving Images based on charac- 
ter/relation using a multimedia content DS such as the 
above-mentioned weight OS and event/object relational 
graph and displaying the same, the method for display- 
ing moving images based on character/relation is char- 
acterized in that: characters are arranged in the order 
of weight information on objects described by a weight 
DS Included in the above multimedia content DS, rela- 
tional transition between two characters are arranged in 



the order of weight information on the relations between 
objects described by a weight DS included in the above 
event/object relational graph DS. and key frames of 
event segments satisfying the relation between the se- 
5 lected two characters are arranged in the order of weight 
information on segments described by a weight DS in- 
cluded in the DS. 

[0076] The method for displaying moving images 
based on character/relatton will now be descrtk>ed with 

10 reference to Figures 9A and 98. 

[0077] Generally, since the contents of a moving im- 
age is displayed based on the relation between main 
characters, a method for indexing and browsing moving 
images based on the relation between characters cart 

15 be usefully utilized. 

[0078] Such a utilization Is made possible using ob- 
jects(characters) and a event/objects relational graph 
DS included in the multimedia content DS. 
[0079] A basic user interface of such a browser is il- 

20 lustrated in Figure 9. 

[0080] In Figure 9, a character screen 901 arranges 
characters appearing in the contents of moving images, 
and a character/relation screen 902 arranges charac- 
ters related with the characters selected in the character 

25 screen 901 and at the same time describes changes in 
the relation between the selected two characters. 
[0081 ] A main scene screen 903 arranges key frames 
representing an event segment satisfying the specific 
relation between these characters. 

30 [0082] At this time, the order of arranging the charac- 
ters arranged in the character screen 901 is set by 
means of the weight DS Included in the content DS. 
[0083] That is. the characters can be arranged in the 
order of priority using weight information on the objects. 

35 [0084] In addition, the order of displaying the relations 
between the characters described in the character/rela- 
tion screen 902 can be determined using the weight DS 
included in the event/object relational graph DS. 
[0085] Finally, the order of key frames satisfying the 

40 selected relation is determined using the weight DS in- 
cluded in the content DS describing weight information 
on segments. 

[0086] As described above. In a structure for multime- 
dia content-based indexing and browsing, the present 

45 invention enables higher-performance indexing and 
browsing by describing effective weight information on 
each content DS. In addition, the present invention pro- 
vides a weight DS corresponding to a complex multime- 
dia content DS in various ways, for thereby preparing a 

50 multimedia standard for multimedia indexing and brows- 
ing using weight information. 

[0087] As the present invention may be embodied in 
several forms without departing from the spirit or essen- 
tial characteristics thereof, it should also be understood 
55 that the above-described embodiments are not limited 
by any of the details of the foregoing description, unless 
othenwise specified, but rather should be construed 
broadly within its spirit and scope as defined in the ap- 
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pended claims, and therefore all changes and modifica- 
tions that fall within the meets and bounds of the claims, 
or equivalence of such meets and bounds are therefore 
intended to be embraced by the appended claims. 



Claims 

1 . A multimedia description scheme, characterized in 
that: each description scheme representing content 
information of multimedia comprises a weight infor- 
mation type meaning the criteria for defining a 
weight value, weight information according to the 
criteria defined by the weight information type, and 
reliability which means how weight information as- 
signed to a target is reliable according to the weight 
information type. 

2. The multimedia description scheme of claim 1, 
wherein the same target can have a plurality of 
weight information according to its criteria using the 
weight information type meaning the criteria for de- 
fining a weight value. 

3. The multimedia description scheme of claim 1, 
wherein the weight Information is described in a n- 
dimensional vector, if the order of weighting targets 
is previously defined. 

4. The multimedia description scheme of claim 1, 
wherein the weight information is described as a set 
of pairs of a reference for weighting target and a 
weight value, if the order of weighting values is not 
previously defined. 

5. A multimedia description scheme for enabling ef- 
fective indexing and browsing, characterized in 
that: weight information on segments or weight in- 
formation on segment relations is described in such 
a manner that a multimedia content description 
scheme includes a segment description scheme 
and a segment relational graph description 
scheme, said multimedia content description 
scheme further including more than one weight de- 
scription scheme. 

6. The multimedia description scheme of claim 5. 
wherein, when the segment description scheme de- 
scribe a hierarchical segment structure including 
another child segment description scheme, one 
weight description scheme included in the multime- 
dia description scheme describes weight informa- 
tion on all segments described in the hierarchical 
segment structure, or a plurality of weight descrip- 
tion schemes each describe weight information in 
unit of child segments of one segment included in 
the hierarchical segment structure. 



7. The multimedia description scheme of claim 5, 
wherein when a multimedia content description 
scheme has a segment relational graph description 
scheme consisting of more than one segment rela- 

5 tion description scheme and more than one seg- 
ment node description scheme, said one segment 
relation description scheme including more than 
one segment relation description scheme or more 
than one segment node, said one segment node de- 

10 scription scheme including more than one segment 
or segment node description scheme, or more than 
one segment relation description scheme, the seg- 
ment relational graph description scheme de- 
scribes weight information on segment relations or 

15 weight infomnation on segment nodes by further in- 
cluding more than one weight description scheme, 
and the segment node description scheme de- 
scribes weight information on segments by further 
including more than one weight description 

20 scheme. 

8. A multimedia description scheme for enabling ef- 
fective multimedia indexing and browsing, charac- 
terized in that: weight information on objects, weight 

25 information on events, or weight information on 
event/object relational graphs are described in such 
a manner that a multimedia content description 
scheme includes a content description scheme con- 
sisting of a plurality of event infomnation. a plurality 

-30 of object information, and a plurality of event/object 
relational graph description schemes, said content 
description scheme further including more than one 
weight description scheme. 

S5 9. The multimedia description scheme of claim 8, 
wherein, when the event or object description 
scheme describes a hierarchical object or event de- 
scription scheme by including another child object 
event description scheme, one weight description 

40 scheme included in the content description scheme 
describes weight information on all objects or 
events described in the hierarchical object or event 
description scheme. 

45 10. The multimedia description scheme of claim 8, 
wherein, when the event or object description 
scheme describes a hierarchical object or event de- 
scription scheme by including another child object 
event description scheme, a plurality of weight de- 

50 scription schemes included in the content descrip- 
tion scheme each describe weight information in 
unit of child object(event) of one object(event) in- 
cluded in the hierarchical object or event description 
scheme. 

55 

11. A multimedia description scheme for enabling ef- 
fective multimedia indexing and browsing, charac- 
terized in that: weight information on relations be- 
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tween events and objects or weight information on 
event or object nodes is described in such a manner 
that a multimedia event/object relational graph de- 
scription scheme further includes more than one 
weight description scheme, and weight information 
on events or objects is described in such a manner 
that an event or object node description scheme fur- 
ther includes more than one weight description 
scheme. 

12. In displaying a key frame transition graph using the 
above-described weight description scheme, multi- 
media content description scheme, and segment 
relational graph description scheme, a method for 
displaying a multimedia summary including the mul- 
timedia description scheme, comprising the steps 
of: 
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IS 



scheme included in the above event/object relation- 
al graph description scheme, and key frames of 

event segments satisfying the relation between the 
selected two characters are arranged in the order 
of weight information on segments described by a 
weight description scheme included in the descrip- 
tion scheme. 



extracting key frames as much as an user 
wants in the order of weight inf onmation on seg- 
ments described in the weight description 
scheme in the multimedia content description 
scheme; and 

extracting only relations including the key 
frames extracted in the segment relational 
graph description scheme, and generating a 
key-frame transition graph using the same. 

13. In displaying a key-frame transition graph using the 
above-described weight description scheme, multi- 
media content description scheme, and segment 
relational graph description scheme, a method for 
displaying a multimedia summary including the mul- 
timedia description scheme of the present inven- 
tion, comprising the steps of: 

extracting segment relations in the order of the 
level of weight information on segment rela- 
tions described in the weight description 
scheme in the segment relational graph de- 40 
scription scheme until the number of key 
frames that the user wants is reached: and 
generating a key-frame transition graph using 
the extracted segment relational graphs. 

45 

14. In indexing moving images based on character/re- 
lation using a multimedia content description 
scheme such as the above-mentioned weight de- 
scription scheme and event/object relational graph 
and displaying the same, a method for displaying so 
moving images based on character/relation is char- 
acterized in that: characters are arranged in the or- 
der of weight information on objects described by a 
weight description scheme Included in the above 
multimedia content description scheme, relational ss 
transition between two characters are arranged In 
the order of weight information on the relations be- 
tween objects described by a weight description 
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FIG. 2 
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FIG.3A 

<Syntact icStructureOS id="#1 "href="ht tp://www. tenip.coin"> 

<WeightDS icl="segment_weightO" type="manuarrel iabi I i ty="1 .0"> 
<We i gh t Va I ueDS Re f e r en ce= " segO " va I ue= " 0. 6 " /> 
<WeightValueDS Reference="seglO"value=''0.7V> 
<We i gh t Va I ueDS Refer ence= " seg 1 1" va I ue= " 0 . 3 " / > 
<We i gh t Va I ueDS Re f e r ence= " seg 1 00 " va I ue= " 0 . 8 V> 
<We i gh t Va I ueDS Refer ence= " segl 0 1" va I ue= " 0 . 3 V > 
<We i gh t Va I ueDS Ref erence=" seg1 1 0" va I ue=" 0 . 2" /> 
<We i gh t Va I ueDS Re f er ence= " seg 1 1 1 " va I ue= " 0 . 5 V> 

</WeightDS> 
</Syntact icStructureDS> 

FIG.3B 

<Syntact icStructureDS id="#1 "href="ht tp://www. temp.com"> 

<WeightDS id^'weight^segO" type="nianuar rel iabi I i ty="1,0-> 
<WeightValueDS Reference="seg10"value="0.6V> 
< We t gh t Va I ueDS Re f e r ence= " seg 11 " va I u e= " 0 . 4 " / > 
</WeightDS> 

<We i gh t DS i d= - we i gh t ^segl 0 " t y pe= " manua I " r e I i ab i I i t y = " 1 . 0 " > 
<We i gh t Va I ueDS Re f er ence= " segl 00 " va I ue= " 0 . 3 V> 
<We I gh t Va I ueDS Refer ence= seg 1 01" va I ue= 0 . 7 " /> 

</WeightDS> 

<WeightDS id^'weight.segir type=*'nianuar rel iabi I i ty="1 .0"> 
<We i gh t Va I ueDS Re f e r ence= " segl 1 0 " va I ue= " 0 . 3 " / > 
<We i gh t Va I ueDS Re f e r ence= " seg 1 1 1" va I ue= " 0 . 7 '7 > 

</WeightDS> 
</Syntact icStructureDS> 
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<l Definition of SyntaticOS with iYeighlOS _!> 

< ! _###########^?#################################_ i > 

<OSType name="Syntact icDS"model="ref inable"> 

<attrDecl name='' IO"><datatypeRef name=''IOV> </attrOecl> 
<attrDecl name="href "xdatatypeRef nafne="ur i **/> </attfDecl> 
<DSTypeRef nanie="WeiQhtDS" niinOccur=''0" maxOccur="*V> 
<DSTypeRef nanie="SegmentDS" minOccur="0'' maxOccur="*"/> 
<DSTypeRef naine="Segment-Rel at ion-Graph" minOccur="0" fliaxOccur="- V> 

</DSType> 
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<i Definition of SegmentRelat lonGraphOS with WeightOS _> 

<OSType name= ' Segmen t^En t i t y-Node "> 

<subOSOf naine= Entity-Node/ > „ . .. , 

<OSType name=''WeightDS" minOccur="0" maxOccur= * /> 
<seQuence minOccur=M"maxOccur="*"> „ 

<choice minOccur="0"(naxOccur= • > 

<OescTypeRef naine= Refer ence-to-Segment /> 

OSTypeRe f naine= ' Segmen t " /> 

<OSTypeRef nanie='Segment-Enti ty-Node /> 

</cho i 

<DSTypeRef name="Seoment-Relat ion"niinOccur="0"(naxOccur="*V> 
</sequence> 

</DSType> > 

<OSTy peRe f nanie= " Segmen t -Re I a t i on " > 

<subDSOf name="ReIation/"> ^ 

<choice ininOccur=*1"maxOccur= * > ^ , ^. . 

<DSTypeRef name= Segment -Re! at ion /> 

<DSTypeRe f name= " Segmen t -En t i t y-Node " /> 

</choice> 
</OSType> 

<DSTypeRe f name= " Segmen t -Re I a t i on-G r aph ' > 
<subDSOf name="ReIation-6raph/"> 
OSTypeRef name="WeightDS"mlnOccur= 0 maxOccur = 
<choice minOccur="1"maxOccur="*"> ^ , ^. 

OSTypeRef name= " Segmen t -Re I a 1 1 on 

<OSTypeRe f name= " Segmen t -En t i t y-Node " /> 

</choice> 
</DSType> 



/> 
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FIG.6A 
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< ! _#############################################_ ! > 
<!_Def ini tion of Semantic DS with Weight OS _!> 
< ! _############################################^#_ i > 

OSType name="Semant icDS" modeI="ref inable'*> 

OSType naine="WeightDS" ininOccur="0" maxOccur ="*"/> 

<DSType name="EventDS" minOccur="0'' maxOccur="« V> 

OSType name="0bjectOS" ininOccur="0'' niaxOccur=''»'7> 

<DSType name="Event/ReoionRelat ionGraphOS" minOccur=''0" maxOccur 

</DSType> 
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FIG.7B 



< ! _##########################################################_> 
<!_Def ini tion of EventObjectRelat ionGraphDS with Weight OS _> 
< ! _##########################################################_> 

<OSType name="Event-Ob ject-Ent i ty-Node"> 
<subDSOf name="Enti ty-Node/"> 

<DSType name="WeightOS" minOccur="0^' maxOccur="*"/> 

<sequence m i nOccu r = " 1 " maxOccu r = " * " > 

<cho i ce mi nOccu r = " 0 " maxOccu r =" * " > 

<OescTypeRef name= "Refer ence- 1 o-E ven t " / > 
<DescTypeRef na[ne=''Ref erence-to-Object "/> 
<OSTy peRe f name= " Even t " /> 

<DSTypeRef name="Object V> 

<DSTypeRef name= " Event -Object -En t i ty-Nocle'7> 

</choice> 

<DSTypeRef name= " Event-Object -Re la t ion'minOccur="0"inaxOccur 
</sequence> 
</DSType> 

<DSTypeRef name="Event-0&ject-Relat ion"> 

<subOS0f name="Relat jon/"> 

<choice minOccur=''1 ''fnaxOccur="*"> 

<DSTypeRef naine="£vent-Object -Relet ion"/> 
<DSTypeRef name="Event-Object-Ent i ty-Node'7> 

</choice> 
</DSType> 

<OSTypeRef nafne="Event-Object-Relat ion-Graph"> 

<subDSO f name= " Re I a t i on-G r aph/ " > 

<OSTy peRe f name= - We i gh t DS'in i nOccu r = "0 "maxOccur = " * " /> 

<choice minOccur="rmaxOccur="*"> 

<DSTypeRef name="Event-Object-Relat ion'7> 
<DSTypeRef name="Event-Object-Ent i ty-Node"/> 

</choice> 
</DSType> 
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